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1. INTRODUCTION 

Advanced modelling and simulation techniques are crucial in analysing the electrical performances 
of complex electronic packages and other interconnections in order to achieve a first-pass design success [I]. 
However, computer-aided design (CAD) tools of high-speed circuit components typically require the develop- 
ment of specific codes or algorithms for the simulation process. These conventional ways to simulate circuits 
are computationally expensive and time consuming, which slows down the design and development process, in 
addition to requiring specific knowledge and engineering know-hows to run and produce the required results. 
On top of that, the simulation of the design has to be repeated whenever there are any changes to the simulation 
parameters, which further adds to the development time. 

Machine learning algorithms are efficient alternatives to conventional methods such as full-wave elec- 
tromagnetic (EM) simulators, which are very accurate but computationally very slow. While analytical methods 
and empirical models are much faster, they are not easy to obtain for new devices, and their accuracy and range 
could be limited. In the past two decades, machine learning models have been used in the modelling of various 
radio frequency (RF) and microwave structures such as antennas and microwave filters [2]-(3]. A recent re- 
view of machine learning techniques on RF and microwave circuit modelling can be found in [4]. The working 
principle of machine learning models is that they can be trained to learn the behaviour of individual circuit com- 
ponents, and then the trained model can be used for high-level design. Normally, the machine learning models 
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are much faster during the predicting than training. Thus, the circuit designers will often spend a considerable 
amount of time to train the machine learning models. However, machine learning models often require a large 
amount of data to be properly trained, and the generation of these data can be very time consuming, especially 
if they are generated using full-wave EM simulators, physics-based models, or measurements [5]. Thus, it is 
beneficial to have machine learning models which can be easily and quickly trained with a limited amount of 
data. 

The artificial neural network (ANN) is one of the most popular machine learning methods for circuit 
modelling. Besides being used for various microwave and RF circuit modelling [6]-[7], the ANN has also 
been used for various signal integrity modelling tasks involving high-speed channels [8], [9]. These include 
modelling of crosstalk {10J, eye diagram [T1]-(12], and transient responses [13], [14]. The Gaussian process 
regression (GP) model is another alternative to the ANN. Similar to the ANN, the GP has also been used for 
the modelling of various microwave structures such as the microwave filter [15], microwave antenna [16], and 
microwave amplifier [17]. Inspired by the success in the implementations of ANN and GP in the field of circuit 
modelling, this paper aims to demonstrate the use of GP and multilayer perceptron (MLP) neural networks 
for the modelling of transmission lines based on their impulse responses. Section 1 of this paper describes 
the introduction and motivation behind this work. Section 2 describes the fundamentals of MLP and GP. 
Section 3 describes the time domain modelling of trainsmission lines, and the motivation behind the utilization 
of the impulse response. The training process of the machine learning models are described in section 4, and 
the results and discussions are presented in section 5. Finally, conclusions and future works are proffered in 
section 6. 


2. MULTILAYER PERCEPTRON AND GAUSSIAN PROCESS 

The MLP is a popular neural network structure consisting of multiple connected network layers [18]. 
Figure 1 shows an MLP neural network with LF layers. It consists of an input layer, an output layer, and one or 
more hidden layers, where each layer has NV; neurons. The MLP receives the inputs, 7 = {@1,%2,%3,..., 2, } 
and generate the outputs, y = {y1, Y2,43,---,yn,}- For many modelling problems, a simple MLP neural 
network consisting of a single hidden layer is sufficient, thanks to the basic characteristics of neural networks, 
which include inherent parallelism, fault tolerance, and learning and self-adaptability, which allows them to 
model even highly non-linear functions [19]. Sometimes, two hidden layers will be used for more complicated 
problems. Multiple papers have been published on MLP neural network applications, for example on fingerprint 
image region of interest segmentation [20], high-electron-mobility transistor (HEMT) modelling [21], and 
the modelling of terminal electrical noise in semiconductor lasers [22]. The basic feature of an ANN is the 
capability to learn the patterns of the input samples to predict the system behaviour. Once the network has 
learnt the input-output relationships, it can then produce outputs for other inputs, even those that were not 
encountered during training. In machine learning terms, this is known as generalization [23]. In the context 
of ANN, the term learning refers to the method of varying the weights of the connecting links between the 
neurons of the network to match to the desired output. Further background on the MLP can be found in [24]. 


Input Layer Hidden Layer Output Layer 


Figure 1. The MLP structure 


The GP is a stochastic modelling methodology that is often used for modelling spatially correlated 
measurements. GP can be succinctly described by 
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f(x) ~ GP (u(x), k (x, x’)) (1) 


where the function is completely defined by its mean function ju(x) and covariance function k (x, x’) [25]. In 
other words, GP can be viewed as a generalization of the multivariate Gaussian probability distribution where 


the evaluations for the inputs x,,..., x), are normally distributed as 
f (x1) 
~ N(u, Kk) (2) 
Ff (xn) 
in which 
Le (X1) 
pea 3 (3) 
L(x) 
and 
k (x1, X1) vee k (x1, Xn) 
k= : : (4) 
k (xn, X1) aves k (xn, Xn) 


The covariance function and mean function play a significant role in a GP-based model. On the other 
hand, the structure of an MLP model will influence the performance of the model. Several procedure have been 
implemented in this project to increase the accuracy of the models. Optimization of hyperparameters are done 
to cope with the real-world non-ideal problem. The hyperparameters for the covariance function and mean 
function in the GP models are also optimized. Furthermore, the structure or hidden units of the MLP models 
are investigated to ensure that the best performing model is selected. 


3. TIME DOMAIN IMPULSE RESPONSE MODELLING OF TRANSMISSION LINES 
Transmission lines are interconnects that relay signals from the transmitters to the receivers on high- 
speed electronic circuits. They are formed by a combination of conductors and dielectric materials to guide a 
signal along the desired path. Generally, the characteristic impedance, or natural impedance Z is the main pa- 
rameter of a transmission line as it defines the behaviour of the line. The characteristic impedance is calculated 


as: 
R+jwL 

Z=,/——— 5 

G+jwC ©) 


where R, L, G, and C are the per unit length series resistance ((Q/m), series inductance (H/m), shunt conduc- 
tance (Q~'/m), and shunt capacitance (F/m) respectively. The physical dimensions of the printed circuit board 
traces or the transmission lines play a significant role in the effects of the characteristic impedance as they 
alter the values of the electrical parameters. For example, the capacitance, C’, for a parallel plate structure is 
calculated as: 


cA 
C= 7 (6) 


where d is the substrate height, A is the area of the trace, and ¢ is the dielectric constant of the surrounding 
material. 

Studying the behaviour of an electronic circuit system is essential before training a machine learning 
model to learn the behaviour of the system. The Gaussian impulse response is the output voltage generated 
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by the electronic circuit system with a Gaussian impulse signal as the input. The Gaussian signal in the time 
domain is selected as the input since the Gaussian transient response represents the behaviour of the electronic 
circuit system under various frequencies. It can be shown that the Fourier transform of a Gaussian signal is 
another Gaussian signal [26], where the Fourier transform is a well known tool that can be used to decompose 
a function or signal into its constituent frequencies. Thus, it is possible to predict the behaviour of the circuit 
over those frequencies given its Gaussian impulse response. 

In this work, circuit modelling of transmission lines are performed using their Gaussian impulse re- 
sponses. Firstly, frequency domain simulations are performed on transmission lines with different design pa- 
rameters to extract their S-parameters. Then, circuit simulations using these S-parameters under a Gaussian 
excitation are performed to extract the impulse responses of the transmission lines. The voltage source, S (t) 
that acts as the input is defined as: 


0 eee oe 
oVv2n 


(7) 


where ju represents the mean and o represents the standard deviation that defines the bandwidth of the Gaussian 
signal. The circuit simulation setup is shown in Figure 2. The design parameters of the transmission line 
considered in this work are: line length, / from 1000 mils to 3700 mils and line width, w from 50 mils to 
110 mil. Figure 3 shows the impulse responses of two randomly selected transmission lines cases, where 
Figure 3(a) shows test case 1 and Figure 3(b) shows test case 2. 


Figure 2. Circuit simulation setup and transmission line parameters 
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Figure 3. The impulse response, V,..4(t) of (a) test case one and (b) test case two 
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4. TRAINING OF MACHINE LEARNING MODELS 


The training process of a MLP model involves searching for a set of weights and biases that results 
in good generalization. The number of neurons and their activation functions are also optimized. In this work, 
the training is done using the Levenberg-Marquardt backpropagation algorithm [27]. MLP models consisting 
of one hidden layer and varying numbers of hidden neurons are trained. Grid search is performed for the 
purpose of hyperparameters optimization, which involves sweeping the number of hidden neurons from 10 to 
100 with a step of 10, and each configuration is trained 10 times to minimize the random nature of training. 
The performances of the models are ranked based on their coefficient of determination, or R-squared values. 


In contrast, for a set of training data, a GP model produces a distribution of multiple possible functions 
with an optimized weight vector, and not just one single function. GP models are specified by their mean and 
covariance functions, but these functions cannot be fully specified a priori. Instead, they form a set of hyper- 
parameters, that is, parameters which have to be determined during training. In this work, first, the declaration 
and initialization of the covariance function, mean function, and their hyperparameters are done. Then, the 
hyperparameters are optimized with the aid of the Gaussian processes for machine learning (GPML) toolbox 
[28]. The optimization is done by minimizing the negative log marginal likelihood together with the partial 
derivatives with respect to the hyperparameters. Finally, the optimized hyperparameters are implemented in 
training the GP model as well as predicting the unseen inputs. 


5. | RESULTS AND DISCUSSION 


In many circuit modelling applications, the generation of the training data can be a very computa- 
tionally expensive process, as each data requires an intricate simulation setup and solution process. Thus, it 
is highly beneficial to have models which are able to function based on a small or limited data sample. In 
this work, the data of 70 transmission line designs are utilized and the data is divided into subsets for train- 
ing, validation, and testing with a ratio of 50:10:10. The training results of the MLP models are tabulated in 
Table 1. By comparing the R-squared values, the model with 50 hidden neurons is selected as the best perform- 
ing model since the model has the greatest R-squared values. 


Table |. Training results of the MLP models 


No. ofneurons TrainingR?  ValidationR2 ‘Testing R2 Training time (s) 


10 0.9276 0.9286 0.9115 9.001 
20 0.9347 0.9154 0.9399 25.605 
30 0.9265 0.9274 0.9288 11.498 
40 0.9255 0.9379 0.9346 23.713 
50 0.9408 0.9540 0.9455 29.734 
60 0.9307 0.9384 0.9371 72.201 
70 0.9367 0.9196 0.9227 39.845 
80 0.9320 0.9278 0.9366 50.927 
90 0.9326 0.9352 0.9142 75.359 
100 0.9277 0.9352 0.9289 82.341 


Table 2 compares the performances of the MLP and GP models. The results show that the GP model 
outperforms the MLP model in both training and testing accuracies. Besides that, the results also show that the 
training of the MLP model is more arduous than the GP model. However, the MLP model can predict much 
faster than the GP model. The GP model shows a higher level of accuracy than the MLP model when both of 
them are trained on small amount of samples, which means that the GP is a very data efficient model. However, 
the slow prediction speed of GP can be a limitation when modelling large amount of designs. Other than that, 
further increasing the amount of training samples can also slow down the prediction speed of the GP model 
even more, whereas the prediction speed of the MLP is not affected by the size of training samples as long 
as the model complexity remains the same. Nevertheless, the training time of the MLP will increase with the 
increase in the size of the training dataset. Figure 4 shows the comparison of the actual impulse responses and 
the MLP predictions, where Figure 4(a) shows the result for test case one and Figure 4(b) shows the result for 
test case two. Figure 5 shows the comparison of the actual impulse responses and the GP predictions, where 
Figure 5(a) shows the result for test case one and Figure 5(b) shows the result for test case two. 
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Figure 4. The impulse response, V,.(t) simulated by the circuit simulator versus MLP predictions of the 
(a) test case one and (b) test case two 


400 


300 


Voltage / mV 


Test Case 1 


x 


Predicted 


Truth 


-100 
0) 


05 1.0 
Time / nSec 
(a) 


15 20 25 30 35 4.0 


300 


ine] 
So 
oO 


Voltage / mV 
3S 
C= | 


-100 


Test Case 2 


Predicted 
x Truth 


* 


x 


0.5 1.0 


15 20 25 30 35 4.0 
Time / nSec 


(b) 


Figure 5. The impulse response, V,,,;(t) simulated by the circuit simulator versus GP predictions of the 


6. CONCLUSION 


Table 2. Comparisons between the MLP and GP models 


(a) test case one and (b) test case two 


Model 
Performance metric MLP GP 
Training MSE 54.29 2.43 x 10-3 
Testing MSE 83.09 22.21 
Average training time (s) 29.7 6.9 
Average prediction time(s) 2.18 x 107? 1.48 


In conclusion, both the MLP and GP models show promising results on the time domain impulse re- 
sponse modelling of transmission lines. On one hand, the GP model shows better performance (lower training 
and testing errors) when trained on a small dataset, while the MLP model has a faster prediction speed. De- 
spite that, we believe that the high data efficiency of the GP model far outweighs its shortcomings, especially 
when each training sample is very expensive to produce. As a future work, the size of the design space can 
be expanded to include increasing amounts and ranges of design parameters in order to better model more 
complicated problems. In this case, the data efficiency of the models will play an important role in optimizing 


the modelling process. 
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